1. Field
Certain aspects of the present disclosure generally relate to neural system engineering and, more particularly, to a method and apparatus for adaptive structural delay plasticity in spiking neural networks.
2. Background
Neurons of a neural network may receive potential inputs from one or more other neurons in the network. The relative importance of inputs may be learned so that only some potential inputs may become active synapses. However, as the number of potential inputs (neurons) in a circuit or network increases, the processing and memory access requirements may increase significantly. For example, if there are M neurons each with N≦M possible inputs, there may be at least M×N potential synapses. Moreover, if connections (synapses) have variable time delays (due to varying connection distance), the number of potential inputs per neuron may be multiplied by a number of possible different time delays (e.g., from 0 to T at time resolution dt yielding R=T/dt possibilities for a total of M×N×R potential synapses for the network). It should be noted that a synapse delay might range from 0 to 25 ms or even higher. Since neuron spike timing precision of a millisecond or far less (e.g., 0.1 or 1 microsecond) may be required, this may increase already significant processing (time) and memory requirements dramatically (e.g., by 250 or 25,000 times, respectively). Therefore, a computational solution of reduced complexity for machine learning is desired.
Biological neural networks may solve this problem with structural plasticity (dendritic or axon growth or retraction, spine or bouton turnover, or spine motility). However, methods to solve this problem in neural system engineering are still unknown.